

<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>User Guide &mdash; QDMA Linux Driver 2018.3 documentation</title>
  

  
  

  

  
  
    

  

  
  
    <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
  

  

  
        <link rel="index" title="Index"
              href="genindex.html"/>
        <link rel="search" title="Search" href="search.html"/>
    <link rel="top" title="QDMA Linux Driver 2018.3 documentation" href="index.html"/>
        <link rel="next" title="User Applications" href="user-app.html"/>
        <link rel="prev" title="QDMA Linux Driver UseCases" href="qdma_usecases.html"/> 

  
  <script src="_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search">
          

          
            <a href="index.html" class="icon icon-home"> QDMA Linux Driver
          

          
          </a>

          
            
            
              <div class="version">
                2018.3
              </div>
            
          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
                <p class="caption"><span class="caption-text">Table of Contents</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="features.html">QDMA Features</a></li>
<li class="toctree-l1"><a class="reference internal" href="system-requirements.html">System Requirements</a></li>
<li class="toctree-l1"><a class="reference internal" href="build.html">Building and Installing Software Stack</a></li>
<li class="toctree-l1"><a class="reference internal" href="devguide.html">Developers Guide</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">User Guide</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#system-level-configurations">System Level Configurations</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#instantiates-the-virtual-functions">1. <strong>Instantiates the Virtual Functions</strong></a></li>
<li class="toctree-l3"><a class="reference internal" href="#allocate-the-queues-to-a-function">2. <strong>Allocate the Queues to a function</strong></a></li>
<li class="toctree-l3"><a class="reference internal" href="#reserve-the-queues-to-vfs">3. <strong>Reserve the Queues to VFs</strong></a></li>
<li class="toctree-l3"><a class="reference internal" href="#set-interrupt-ring-size">4. <strong>Set Interrupt Ring Size</strong></a></li>
<li class="toctree-l3"><a class="reference internal" href="#set-completion-interval">5. <strong>Set Completion Interval</strong></a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#queue-management">Queue Management</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="user-app.html">User Applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="performance.html">QDMA Performance</a></li>
</ul>

            
          
        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
        <a href="index.html">QDMA Linux Driver</a>
      </nav>


      
      <div class="wy-nav-content">
        <div class="rst-content">
          

 



<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href="index.html">Docs</a> &raquo;</li>
      
    <li>User Guide</li>
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="_sources/userguide.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="user-guide">
<h1>User Guide<a class="headerlink" href="#user-guide" title="Permalink to this headline">¶</a></h1>
<p>This section describes the details on controlling and configuring the QDMA IP</p>
<div class="section" id="system-level-configurations">
<h2>System Level Configurations<a class="headerlink" href="#system-level-configurations" title="Permalink to this headline">¶</a></h2>
<p>QDMA driver provides the sysfs interface to enable to user to perform system level configuations. QDMA <code class="docutils literal notranslate"><span class="pre">PF</span></code> and <code class="docutils literal notranslate"><span class="pre">VF</span></code> drivers expose several <code class="docutils literal notranslate"><span class="pre">sysfs</span></code> nodes under the <code class="docutils literal notranslate"><span class="pre">pci</span></code> device root node.</p>
<p>Once the qdma module is inserted and until any queue is added into the system and FMAP programming is not done, sysfs provides an interface to configure parameters for the module configuration.</p>
<p>[xilinx&#64;]# lspci | grep -i Xilinx</p>
<div class="line-block">
<div class="line">01:00.0 Memory controller: Xilinx Corporation Device 903f</div>
<div class="line">01:00.1 Memory controller: Xilinx Corporation Device 913f</div>
<div class="line">01:00.2 Memory controller: Xilinx Corporation Device 923f</div>
<div class="line">01:00.3 Memory controller: Xilinx Corporation Device 933f</div>
</div>
<p>Based on the above lspci output, traverse to <code class="docutils literal notranslate"><span class="pre">/sys/bus/pci/devices/&lt;device</span> <span class="pre">node&gt;/qdma</span></code> to find the list of configurable parameters specific to PF or VF driver.</p>
<div class="section" id="instantiates-the-virtual-functions">
<h3>1. <strong>Instantiates the Virtual Functions</strong><a class="headerlink" href="#instantiates-the-virtual-functions" title="Permalink to this headline">¶</a></h3>
<p>QDMA IP supports 252 Virtual Functions(VFs). <code class="docutils literal notranslate"><span class="pre">/sys/bus/pci/devices/&lt;device</span> <span class="pre">node&gt;</span></code> provides two configurable entries</p>
<ul class="simple">
<li><code class="docutils literal notranslate"><span class="pre">sriov_totalvfs</span></code> : Indicates the maximum number of VFs supported for PF. This is a read only entry which can be configured during bit stream generation.</li>
<li><code class="docutils literal notranslate"><span class="pre">sriov_numvfs</span></code> : Enables the user to specify the number of VFs required for a PF</li>
</ul>
<p>Display the currently supported max VFs:</p>
<p>[xilinx&#64;]# cat /sys/bus/pci/devices/0000:01:00.0/sriov_totalvfs</p>
<p>Instantiate the required number of VFs for a PF:</p>
<p>[xilinx&#64;]# echo 3 &gt; /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs</p>
<p>Once the VFS are instantiated, required number of queues can be allocated the VF using <code class="docutils literal notranslate"><span class="pre">qmax</span></code> sysfs entry available in VF at
/sys/bus/pci/devices/&lt;VF function number&gt;/qdma/qmax</p>
</div>
<div class="section" id="allocate-the-queues-to-a-function">
<h3>2. <strong>Allocate the Queues to a function</strong><a class="headerlink" href="#allocate-the-queues-to-a-function" title="Permalink to this headline">¶</a></h3>
<p>QDMA IP supports maximum of 2048 queues. By default all the queues are equally distributed to the physical functions and each function gets 512 queues each.</p>
<p><code class="docutils literal notranslate"><span class="pre">qmax</span></code> configuration parameter enables the user to update the number of queues for a PF. This configuration parameter indicates “Maximum number of queues associated for the current pf”.</p>
<p>If the queue allocation needs to be different for any PF, access the qmax sysfs entry and set the required number.</p>
<p>Once the number of queues for any PF is changed from the default value, the remaining set of queues among the 2048 queues are evenly distributed for the remaining PFs.</p>
<p>Display the current value:</p>
<p>[xilinx&#64;]# cat /sys/bus/pci/devices/0000:01:00.0/qdma/qmax</p>
<p>Set a new value:</p>
<p>[xilinx&#64;]# echo 1024 &gt; /sys/bus/pci/devices/0000:01:00.0/qdma/qmax</p>
<p>Ex: Default queue sets for all PFs</p>
<p>[xilinx&#64;]# dmactl dev list</p>
<div class="line-block">
<div class="line">qdma01000     0000:01:00.0    max QP: 449, 0~448</div>
<div class="line">qdma01001     0000:01:00.1    max QP: 449, 449~897</div>
<div class="line">qdma01002     0000:01:00.2    max QP: 449, 898~1346</div>
<div class="line">qdma01003     0000:01:00.3    max QP: 449, 1347~1795</div>
</div>
<p>[xilinx&#64;]# echo 1770 &gt;  /sys/bus/pci/devices/0000:01:00.0/qdma/qmax</p>
<p>[xilinx&#64;]# dmactl dev list</p>
<div class="line-block">
<div class="line">qdma01000     0000:01:00.0    max QP: 1770, 0~1769</div>
<div class="line">qdma01001     0000:01:00.1    max QP: 8, 1770~1777</div>
<div class="line">qdma01002     0000:01:00.2    max QP: 8, 1778~1785</div>
<div class="line">qdma01003     0000:01:00.3    max QP: 8, 1786~1793</div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">qmax</span></code> configuration parameter is available for virtual functions as well.  Once the <code class="docutils literal notranslate"><span class="pre">qmax_vfs</span></code> is configured, qmax for each VF can be updated from pool of queues assigned for the VFs.</p>
</div>
<div class="section" id="reserve-the-queues-to-vfs">
<h3>3. <strong>Reserve the Queues to VFs</strong><a class="headerlink" href="#reserve-the-queues-to-vfs" title="Permalink to this headline">¶</a></h3>
<p>QDMA IP supports 2048 queues and all the queues are allocated to PFs by default. From the set of 2048 queues, <code class="docutils literal notranslate"><span class="pre">qmax_vfs</span></code> configuration parameter enables the user to allocate a set of queues to the VFs.</p>
<p>This entry is available only for Master PF.</p>
<p>Before instantiating the VFs, allocate required number of queues for VFs from the available pool.</p>
<p>Assume that PF0 is the master PF.</p>
<p>Display the current value:</p>
<p>[xilinx&#64;] #cat /sys/bus/pci/devices/0000:81:00.0/qdma/qmax_vfs</p>
<p>Set a new value:</p>
<p>[xilinx&#64;] #echo 1024 &gt; /sys/bus/pci/devices/0000:81:00.0/qdma/qmax_vfs</p>
</div>
<div class="section" id="set-interrupt-ring-size">
<h3>4. <strong>Set Interrupt Ring Size</strong><a class="headerlink" href="#set-interrupt-ring-size" title="Permalink to this headline">¶</a></h3>
<p>Interrupt ring size is associated with indirect interrupt mode.</p>
<p>When the module is inserted in indirect interrupt mode, by default the interrupt aggregation ring size is set 0 i.e 512 entries</p>
<p>User can configure he interrupt ring entries in multiples of 512 hence set the <code class="docutils literal notranslate"><span class="pre">intr_rngsz</span></code> with multiplication factor</p>
<div class="line-block">
<div class="line">0 - INTR_RING_SZ_4KB, Accommodates 512 entries</div>
<div class="line">1 - INTR_RING_SZ_8KB, Accommodates 1024 entries</div>
<div class="line">2 - INTR_RING_SZ_12KB, Accommodates 1536 entries</div>
<div class="line">3 - INTR_RING_SZ_16KB, Accommodates 2048 entries</div>
<div class="line">4 - INTR_RING_SZ_20KB, Accommodates 2560 entries</div>
<div class="line">5 - INTR_RING_SZ_24KB, Accommodates 3072 entries</div>
<div class="line">6 - INTR_RING_SZ_24KB, Accommodates 3584 entries</div>
<div class="line">7 - INTR_RING_SZ_24KB, Accommodates 4096 entries</div>
</div>
<p>Display the current value:</p>
<p>[xilinx&#64;]# cat /sys/bus/pci/devices/0000:81:00.0/qdma/intr_rngsz</p>
<p>Set a new value:</p>
<p>[xilinx&#64;]# echo 2 &gt; /sys/bus/pci/devices/0000:81:00.0/qdma/intr_rngsz</p>
</div>
<div class="section" id="set-completion-interval">
<h3>5. <strong>Set Completion Interval</strong><a class="headerlink" href="#set-completion-interval" title="Permalink to this headline">¶</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">cmpt_intrvl</span></code> indicated the interval at which completions are generated for an MM or H2C Stream queue running in non-bypass mode.
User can set any of the following list of values for this configuration parameter.</p>
<div class="line-block">
<div class="line">3’h0: 4</div>
<div class="line">3’h1: 8</div>
<div class="line">3’h2: 16</div>
<div class="line">3’h3: 32</div>
<div class="line">3’h4: 64</div>
<div class="line">3’h5: 128</div>
<div class="line">3’h6: 256</div>
<div class="line">3’h7: 512</div>
</div>
<p>Completion accumulation value is calculated as 2^(register bit [2:0]). Maximum accumulation is 512.
Accumulation can be disabled via queue context.</p>
<p>Display the current value:</p>
<p>[xilinx&#64;]# cat /sys/bus/pci/devices/0000:81:00.0/qdma/cmpt_intrvl</p>
<p>Set a new value:</p>
<p>[xilinx&#64;]# echo 2 &gt; /sys/bus/pci/devices/0000:81:00.0/qdma/cmpt_intrvl</p>
</div>
</div>
<div class="section" id="queue-management">
<h2>Queue Management<a class="headerlink" href="#queue-management" title="Permalink to this headline">¶</a></h2>
<p>QDMA driver comes with a command-line configuration utility called <code class="docutils literal notranslate"><span class="pre">dmactl</span></code> to manage the queues in the system.</p>
</div>
</div>


           </div>
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
        <a href="user-app.html" class="btn btn-neutral float-right" title="User Applications" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
      
      
        <a href="qdma_usecases.html" class="btn btn-neutral" title="QDMA Linux Driver UseCases" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2018-2019, Xilinx, Inc.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  


  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'./',
            VERSION:'2018.3',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE:  true
        };
    </script>
      <script type="text/javascript" src="_static/jquery.js"></script>
      <script type="text/javascript" src="_static/underscore.js"></script>
      <script type="text/javascript" src="_static/doctools.js"></script>

  

  
  
    <script type="text/javascript" src="_static/js/theme.js"></script>
  

  
  
  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.StickyNav.enable();
      });
  </script>
   

</body>
</html>